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ARMY  EXCELLENCE  IN  LEADERSHIP  (AXL):  A  MULTIMEDIA  APPROACH  TO 
BUILDING  TACIT  KNOWLEDGE  AND  CULTURAL  REASONING 

EXECUTIVE  SUMMARY 


Research  Requirement: 

The  current  operating  environment  requires  junior  officers  to  rely  upon  diverse  skills. 
Many  junior  leaders  will  encounter  a  wide  spectrum  of  operations  while  deployed,  and  only  a 
portion  of  those  operations  will  draw  upon  their  tactical  expertise.  Junior  leaders  may  be 
required  to  engage  in  several  non-tactical  activities,  such  as  interacting  with  the  civilian 
population,  building  marketplaces,  and  negotiating  with  tribal  leaders.  This  report  describes  a 
computer- administered  leadership  intervention  called  Army  Excellence  in  Leadership  (AXL), 
which  was  designed  to  help  prepare  junior  Army  officers  for  the  complex  challenges  that  they 
might  encounter  in  the  Middle  East.  This  report  describes  preliminary  research  regarding  the 
effectiveness  of  a  cultural  awareness  module  from  the  AXL  system. 

Procedure: 

Fifty-five  junior  officers  from  three  United  States  Army  installations  completed  a  cultural 
awareness  module  in  the  AXL  system.  The  training  module  consisted  of  two  primary 
components.  First,  small  groups  of  officers  watched  a  short  filmed  case  study  about  a  fictional 
mission  in  Iraq.  Second,  the  AXL  system  facilitated  small-group  discussion  of  the  cultural  issues 
and  implications  embedded  in  the  case  study.  Officers  completed  research  measures  at  three 
time  points  during  discussion:  (1)  before  watching  the  film,  (2)  after  watching  the  film,  and  (3) 
after  completing  discussion.  Measures  assessed  various  types  of  emotional  reactions,  reactions 
to  the  film,  reactions  to  the  training  concept,  and  short-term  indicators  of  learning. 

Findings: 

Results  indicated  that  officers  performed  better  on  a  judgment  task  after  having 
completed  training.  Additionally,  cultural  issues  embedded  in  the  scenario  were  more  salient  to 
officers  after  completion  of  the  cultural  awareness  module.  With  respect  to  different  elements  of 
the  AXL  system,  findings  indicated  that  the  film  was  emotionally  evocative  and  the  training 
concept,  in  general,  was  well-received.  Further,  emotional  responses  to  the  film  were  related  to 
learning-relevant  variables,  such  as  judgment  scores  and  officer  reports  that  they  could  apply  the 
training  to  their  activities  as  a  leader. 

Utilization  and  Dissemination  of  Findings: 

This  research  provides  preliminary  evidence  that  the  AXL  system  and  the  multimedia 
design  principles  that  it  embodies  can  assist  junior  officers  develop  culturally  aware  thinking. 
Instructors  and  trainers  interested  in  helping  officers  acquire  tacit  knowledge  can  leverage  some 
of  these  multimedia  concepts  to  their  advantage.  This  investigation  also  suggests  that  affect  may 
play  an  important  role  in  training  and  learning. 
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ARMY  EXCELLENCE  IN  LEADERSHIP  (AXL):  A  MULTIMEDIA  APPROACH 
TO  BUILDING  TACIT  KNOWLEDGE  AND  CULTURAL  REASONING 


Today’s  military  operating  environment  is  demanding  and  complex,  requiring  Army 
leaders  to  draw  from  skill  sets  that  were  unanticipated  during  the  Cold  War  era.  Indeed,  it  is  not 
unusual  for  junior  leaders  (i.e.,  lieutenants  and  captains)  to  deal  with  circumstances  that  would 
have  been  handled  by  their  senior  leader  counterparts  twenty  years  ago.  Many  junior  leaders  will 
encounter  a  wide  spectrum  of  operations  while  deployed,  and  only  a  portion  of  those  operations 
will  draw  upon  their  tactical  expertise.  In  addition  to  establishing  local  security  and  capturing 
insurgents,  junior  leaders  may  be  required  to  engage  in  non-tactical  activities,  such  as  interacting 
with  the  civilian  population,  building  marketplaces,  and  negotiating  with  tribal  leaders.  As 
Major  General  Scales  noted, 

“Reflective  senior  officers  returning  from  Iraq  and  Afghanistan  are  telling  us  that  wars 
are  won  by  creating  alliances,  leveraging  nonmilitary  advantages,  reading  intentions, 
building  trust,  converting  opinions,  and  managing  perceptions,  all  tasks  that  demand 
an  exceptional  ability  to  understand  people,  their  cultures,  and  their  motivation.  While 
wars  have  become  more  complex,  responsibility  for  those  who  fight  them  has 
increasingly  slipped  down  the  chain  of  command  to  junior  personnel.  Yet  these 
young,  inexperienced  leaders  have  little  time  to  prepare  themselves  to  make  strategic 
decisions”  (2006,  pg.  38). 

Because  the  current  operating  environment  places  many  demands  on  junior  leaders,  the 
United  States  Army  has  devoted  substantial  attention  to  the  challenge  of  how  to  help  junior 
leaders  become  adaptive  thinkers,  culturally  aware,  and  interpersonally  competent  (e.g.,  Brown, 
2003;  Burpo,  2006;  Moilanen  &  Craig,  2000;  Petraeus,  2006;  Scales,  2006).  This  paper 
describes  a  leadership  intervention  designed  to  prepare  junior  Army  officers  for  the  complex  and 
chaotic  situations  that  they  will  encounter  in  the  Middle  East  and  presents  preliminary  research 
regarding  the  effectiveness  of  the  intervention  prototype. 


OVERVIEW  OF  THE  ARMY  EXCELLENCE  IN  LEADERSHIP  (AXL)  SYSTEM 

The  Army  Excellence  in  Leadership  (AXL)  system  is  an  online  educational  tool  that 
targets  the  development  of  tacit  leadership  knowledge  and  cultural  awareness  in  junior  leaders. 
The  AXL  system  is  a  variation  of  case  method  instruction  in  which  junior  leaders  are  presented 
with  fictional  case  studies  about  military  leadership  in  a  tactical  context  and  then  asked  to 
analyze,  reflect  on,  and  discuss  the  leadership  issues  embedded  in  the  cases.  The  AXL  approach 
to  case  method  instruction  differs  from  traditional  case  method  instruction  in  at  least  two  ways. 
First,  traditional  case  method  instruction  often  relies  on  paper-based  cases.  AXL  cases  are 
Hollywood-style  films.  Second,  in  the  traditional  case  method  approach,  an  instructor  plays  an 
essential  role  in  facilitating  discussion  of  the  case.  In  AXL,  the  online  system  facilitates 
discussion  among  leaders  and  serves  as  a  proxy  for  a  human  instructor.  These  two  features  of 
the  AXL  system  are  discussed  in  greater  detail  in  the  subsequent  sections. 
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AXL  Case  Studies 


The  idea  of  using  movies  as  case  studies  to  demonstrate  concepts  or  stimulate  critical 
thinking  is  not  new  or  unique  (e.g.,  Cavalier  &  Weber,  2002;  Green  &  Cotlar,  1973a;  Leeper, 
1993;  Richardson,  1994;  Sprau,  2001).  Filmed  case  studies  offer  several  advantages  over 
traditional  paper  cases.  First,  paper  cases  require  a  certain  level  of  reading  proficiency,  while 
film  does  not. 

Second,  film  provides  a  multimedia  experience  that  allows  students  to  see  and  hear  all  of 
the  events  depicted  in  the  case  as  if  they  were  actually  in  the  situation  (Green  &  Cotlar,  1973b). 
Paper  cases  rely  primarily  on  the  reader  constructing  a  mental  representation  of  events  based  on 
written  descriptions.  When  done  correctly,  film  can  achieve  a  higher  level  of  fidelity  to  the 
performance  environment  than  can  a  paper  case.  Film,  therefore,  can  accomplish  a  level  of 
immersion,  complexity,  and  realism  that  might  not  be  achieved  by  paper  cases. 

Third,  a  movie  is  an  audiovisual  experience  containing  dynamic  visual  images  of 
situational  context,  depictions  of  character  activity,  spoken  character  dialogue,  and  background 
sounds.  These  attributes  of  film  not  only  improve  the  fidelity  of  a  student’s  experience;  such 
attributes  also  might  help  students  process  case  information  more  effectively  than  they  would 
process  the  infonnation  contained  in  a  paper  case.  Several  lines  of  research  on  perception, 
memory,  and  learning  provide  strong  evidence  that  individuals  process  verbal  and  visual 
infonnation  through  two  processing  systems  that  operate  somewhat  independently  of  one  another 
(e.g.,  Baddeley,  1986,  2003;  Baddeley  &  Hitch,  1991;  Gehring  &  Toglia,  1988;  Holmes,  Brewin, 
&  Hennessy,  2004;  Mayer,  2001;  Mousavi,  Low,  &  Sweller,  1995;  Paivio,  1986,  1991).  To 
grossly  oversimplify — sounds  are  processed  through  one  system  and  visible  images  are 
processed  through  another.  Although  limitations  exist  with  respect  to  how  much  information 
each  system  can  process  individually,  cognitive  load  as  a  whole  can  be  reduced  by  presenting 
infonnation  in  multiple  modalities  so  that  neither  processing  system  is  unduly  overburdened. 
Mousavi  et  al.  (1995)  indicated  that: 


[BJasic  research  into  the  characteristics  of  working  memory  has  suggested  that  this 
processing  system  is  divided  into  at  least  two  partially  independent  processors:  an  auditory 
system  devoted  heavily  to  language  and  a  visual  system  for  handling  images,  including 
writing.  Because  both  systems  can  be  used  simultaneously,  limited  working  memory  capacity 
might  be  effectively  increased  if  information  that  must  be  stored  or  simultaneously  processed 
is  presented  in  a  manner  that  permits  it  to  be  divided  between  the  two  systems,  rather  than 
processed  in  one  system  alone.  As  a  consequence,  informationally  equivalent  material  that 
may  be  difficult  to  process  in  a  purely  visual  manner  may  be  more  easily  handled  if  it  can  be 
presented  partially  in  both  modalities”  (pg.  331). 


The  two  best  known  dual-processing  theories  are  those  of  Paivio  (1986)  and  Baddeley 
(1986).  More  recently,  Mayer  (2001)  proposed  a  dual-processing  theory  of  multimedia  learning, 
which  suggested  that  presentations  using  multiple  modes  of  information  transmission  are 
superior  to  single  mode  presentations.  Mayer’s  theory  proposed  that  individuals  process 
incoming  information  through  two  separate  channels:  a  visual  channel  that  registers  images  and 
an  auditory  channel  that  registers  sounds.  These  two  channels  operate  as  distinct  processing 
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systems,  and  they  result  in  two  separate  knowledge  representations — a  verbal  representation  and 
a  visual  representation.  In  addition  to  the  visual  and  verbal  representations,  individuals  are  able 
to  form  connections  between  the  two  knowledge  representations,  resulting  in  a  coherent  and 
elaborate  representation  of  knowledge. 

In  sum,  the  accumulated  body  of  research  on  dual-processing  theories  suggests  that 
individuals  are  able  to  process  and  retain  more  information  when  multiple  modalities  are  used 
than  when  only  a  single  modality  is  used.  A  series  of  experiments  conducted  by  Thompson  and 
Paivio  (1994)  exemplifies  this  idea  well.  They  found  that  recall  was  significantly  higher  when 
individuals  were  presented  with  both  pictures  of  objects  and  their  associated  sounds  rather  than 
presented  with  only  pictures  or  only  sounds. 

Within  the  context  of  AXL,  leaders  who  watch  the  filmed  cases  are  exposed  to  rich 
sources  of  both  visual  information  and  audio,  and  they  may  be  more  likely  to  retain  this 
information  than  if  they  are  exposed  to  the  case  information  presented  in  a  single  modality.  This 
improved  probability  of  recalling  relevant  details  from  the  multimedia  scenario  may,  in  turn, 
help  to  foster  more  profound  and  informative  discussion  about  the  case  among  students  during 
the  discussion  phase  of  case  method  teaching. 

The  AXL  system  currently  houses  two  filmed  case  studies,  Power  Hungry  and  Tripwire. 
Both  Tripwire  and  Power  Hungry  are  short  films,  lasting  no  longer  than  15  minutes.  The  films 
were  written,  acted,  directed,  and  edited  by  Hollywood  professionals,  resulting  in  high-quality 
films  similar  to  those  viewed  in  a  movie  theater.  The  first  film,  Power  Hungry,  was  created  for 
the  AXL  system  in  2003.  Power  Hungry  depicts  a  captain  tasked  with  securing  a  site  for  a  food 
distribution  operation  in  Afghanistan.  In  addition  to  poor  interpersonal  skills  and  a  lack  of 
cultural  awareness,  the  captain  is  confronted  with  warlords,  terrain  issues,  time  constraints,  and 
inexperienced  subordinates.  The  mission  ends  in  failure,  with  an  Afghan  warlord  seizing  the 
food  trucks.  A  still  image  from  the  Power  Hungry  film  is  presented  in  Figure  1 . 

Power  Hungry  centers  around  six  teaching  themes:  understanding  the  mission,  sharing 
vision,  command  influence,  establishing  a  model  of  command,  cultural  awareness,  and 
employing  the  expertise  of  non-commissioned  officers  (NCOs).  Previous  research  on  Power 
Hungry  indicated  that  watching  the  film  heightened  the  emotional  arousal  of  junior  leaders.  The 
film  also  was  more  memorable  and  resulted  in  greater  positive  affect  than  the  same  case  study 
presented  in  a  multimedia  PowerPoint  format  (Zbylut  &  Ward,  2004a;  Zbylut,  Ward,  &  Mark, 
2005).  The  potential  for  film  to  engage  students  both  cognitively  and  emotionally  may  benefit 
the  learning  process  by  helping  students  extract  and  retain  important  information  from  the  lesson, 
as  well  as  keeping  students  motivated  throughout  the  lesson. 

The  second  filmed  case  study,  Tripwire,  was  completed  in  2005.  The  Tripwire  film  is  set 
in  Iraq  and  the  captain’s  mission  is  to  arrange  a  meeting  between  two  tribal  leaders.  In  Tripwire, 
the  Army  battery  encounters  several  obstacles  to  accomplishing  its  mission,  including  a  fatal 
attack  on  an  interpreter,  insurgent  activity,  and  improvised  explosive  devices  (IEDs).  Tripwire 
incorporates  many  of  the  same  communication,  leadership,  and  cultural  issues  embedded  in 
Power  Hungry,  but  includes  additional  themes,  such  as  establishing  trust  with  subordinates  and 
taking  care  of  Soldiers.  A  still  image  from  Tripwire  is  presented  in  Figure  2. 
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Figure  1.  In  Power  Hungry,  an  Afghan  warlord  expresses  his  displeasure  to  an  American  Soldier. 


Figure  2.  In  Tripwire,  American  Soldiers  interview  their  new  interpreter. 


Although  many  case  studies  are  based  on  historical  accounts  of  events  that  occurred, 
historical  accuracy  is  not  a  necessary  precondition  for  case  study  development.  For  example, 
Maltby  (2001)  noted  that,  “There  are  two  possibilities  when  designing  cases — to  invent  a 
situation,  or  to  draw  on  a  real  one”  (pg.  424).  Similarly,  Jennings  (1996)  indicated  that  “teaching 
cases  may  not  be  ‘true’  accounts  of  business  life”  (pg.  5),  and  suggested  that  other  important 
criteria  for  determining  the  usefulness  of  a  case  study  include  how  well  the  case  stimulates 
discussion  among  students  and  how  well  the  case  engages  a  student  intellectually  and 
emotionally.  What  appears  to  be  most  important  about  a  case  is  not  necessarily  that  it  is  a  true 
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account,  but  rather  that  it  is  realistic,  complex,  and  built  for  accomplishing  the  teaching  goals  of 
the  lesson.  Along  these  lines,  the  films  embedded  in  the  AXL  system  are  not  historical  accounts. 
Instead,  the  AXL  films  take  an  amalgam  of  real  events  and  military  issues  to  provide  a  rich, 
layered,  and  multifaceted  stimulus  for  prompting  student  discussion  about  teaching  points 
intentionally  embedded  in  the  stories.  Both  Power  Hungry  and  Tripwire  were  built  around 
content  extracted  from  interviews  with  captains  who  had  returned  from  recent  deployments 
(Gordon,  2005;  R.  Hill,  Douglas,  Gordon,  Pighin,  &  van  Velsen,  2003;  R.  Hill,  Gordon,  &  Kim, 
2004).  Challenges,  problems,  and  situations  that  captains  encountered  while  deployed  were 
interwoven  into  the  storylines  of  the  films.  To  ensure  that  the  films  were  realistic,  subject  matter 
experts  (e.g.,  experts  in  military  leadership,  experts  in  IEDs)  reviewed  the  film  scripts  prior  to 
filming.  Department  of  Army  personnel  were  on  location  during  filming  to  ensure  military 
details  were  accurate. 


Interactive  AXL  Modules 

In  traditional  case-method  teaching,  an  instructor  guides  student  discussion  and  analysis 
of  the  case.  Unfortunately,  preparing  for  a  case  study  can  be  both  labor  and  time-intensive 
(Diamantes  &  Ovington,  2003),  and  many  instructors  may  not  have  adequate  time  available  to 
prepare.  Additionally,  the  Army  is  seeking  innovative  ways  to  improve  distance  learning,  which 
requires  that  appropriate  online  proxies  for  human  instructors  be  developed.  The  AXL  system 
seeks  to  address  both  of  these  challenges  by  providing  online  multimedia  exercises  that  will 
prompt  students  to  discuss  and  analyze  the  issues  embedded  in  a  case  study  without  the  aid  of  a 
“live”  instructor. 

Each  film  in  the  AXL  system  is  paired  with  several  interactive  modules  that  address 
military  leadership  topics,  such  as  cultural  awareness,  communicating  with  subordinates,  and 
working  with  non-commissioned  officers  (NCOs).  The  modules  were  intended  to  be  completed 
in  small  groups  of  two  to  four  Soldiers.  Junior  leaders  who  wish  to  learn  about  a  topic  (or  are 
told  to  learn  about  a  topic  by  an  instructor)  select  a  module  related  to  that  issue.  After  selecting  a 
topic,  the  AXL  system  presents  the  goals  of  the  module  to  the  group.  The  group  then  watches 
the  associated  filmed  case  on  the  computer.  Once  the  group  has  watched  the  film,  the  AXL 
system  automatically  transitions  into  the  discussion  phase  of  instruction.  A  screen  shot  from  an 
AXL  online  module  is  presented  in  Figure  3. 

AXL  facilitates  discussion  of  the  filmed  cases  by  using  a  combination  of  close-ended  and 
open-ended  questions.  The  AXL  system  uses  answers  to  close-ended  questions  to  check  for 
understanding  of  basic  concepts,  provide  feedback,  and  branch  the  discussion  to  compel  leaders 
to  examine  their  thoughts  and  assumptions  more  closely.  Open-ended  questions  are  used  to 
stimulate  reflection  and  analytical  thinking,  encourage  the  sharing  and  application  of  real-world 
experiences,  and  generate  solutions  to  problems  in  the  case  study.  Another  feature  of  the  AXL 
system  is  that  leaders  can  interview  the  main  characters  from  a  case  study  to  ascertain  what  the 
characters  knew,  felt,  and  thought  during  the  scenario.  Leaders  type  questions  into  the  AXL 
system  to  reveal  a  character’s  perspective  on  the  scenario  and  uncover  information  not  included 
in  the  film.  Figure  4  depicts  an  example  of  a  group  querying  a  film  character  in  the  AXL  system. 
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Now  that  youVe  nod  me  opportunity  to  review  some  of  me  diverse  issues  embedded  in  T  rtpwire, 
let's  focus  specifically  on  me  topic  of  cultural  awareness. 

in  Tripwire,  the  local  situation  affects  me  battery  and  me  outcome  of  me  scenario  CPT  Holston 
makes  tactical  decisions  regarding  Improvised  explosive  devices  (lEDs)  mat  affect  tne 
perceptions  and  general  state  of  me  local  population  He  also  encounters  challenges  in  dealing 
witn  Interpreters,  first  losing  an  Interpreter  to  gunfire  and  then  acquiring  a  new  Interpreter  wno 
has  not  been  cleared  through  headquarters. 

We  will  now  consider  me  following  issues  as  they  relate  to  cultural  awareness: 

I .  The  local  situation  as  a  factor  in  the  mission 

2  Dealing  with  local  residents 

3  Differences  within  a  single  culture 

Click  Next  to  begin 


Figure  3 .  A  screen  shot  from  a  Cultural  Awareness  module  for  Tripwire . 


I'*J  ftfmy  fwrHImrr  in  I  *<nft»r*hip  -  AW  J»H  « Mhwuoft  lolrrnrl  f>n*nrrr 


fie  Edl  View  F  avert  M  Took  Help  * 

Q  e«;k  *  Q  *  i*)  .;»•  /  vVF4vw,w  0  M  .  j  ££  “S 

Mi-n  Maxhmvw .cl .use .edu/«Vnde* .phcfq-ZS  -  a  GO  IS*-:  *  • 

V/  *  ■•’Search  Web  *  £/  L'l*  d>  Yahoo'  *  Cp  Answers  •  Games  *  ^  lex.  si  »  ?<  HuUt  •  '-^MyVteb"  * 


Figure  4.  Example  of  a  leader  interviewing  one  of  the  characters  from  Tripwire.  The  leader  types  the  question, 
“How  did  you  fiance  affect  your  performance?”  The  character  replies,  “If  you’re  asking  -was  1  bothered  by  finding 
out  my  girlfriend  was  pregnant? — you  bet  I  was.  It  was  all  I  could  think  about.  I  mean,  here  I  am  a  million  miles 
away  and  she  does  what  she  did.  But  when  the  time  came,  I  thought  was  ready.  Wasn’t  I?”  The  character’s  answer 
corresponds  to  the  leadership  issue  of  Taking  Care  of  Soldiers.  Also  note  that  minor  grammatical  errors,  such  as 
using  “you”  instead  of  “your,”  do  not  interfere  with  the  relevance  of  the  reply. 
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Theoretical  Underpinnings  of  AXL 


The  AXL  system  design  and  pedagogical  approach  draws  from  a  variety  of  research 
domains.  First,  AXL  is  a  form  of  case  method  instruction,  which  is  widely  hailed  as  an  effective 
approach  for  developing  critical  thinking,  problem  solving,  and  interpersonal  skills  (e.g., 
Crittenden,  Crittenden,  &  Hawes,  1999;  Hassall,  Lewis,  &  Broadbent,  1998;  Jennings,  1996; 
Kreber,  2001;  Richardson,  1994;  Stewart  &  Dougherty,  1993).  AXL  instruction  includes 
realistic,  multifaceted  cases  and  focuses  on  helping  leaders  learn  “how  to  think”  rather  than 
“what  to  think,”  thereby  preparing  leaders  to  deal  with  complexity  when  they  encounter  it  in 
their  work  environment  (Hannafin,  J.  Hill,  &  Land,  1997;  Jonassen,  2002;  Lesgold,  2001). 
Moreover,  AXL’s  use  of  multimedia  cases  in  conjunction  with  interactive  multimedia  modules 
can  help  leaders  construct  more  elaborate  and  sophisticated  knowledge  representations  than  they 
would  if  information  were  presented  using  a  single  modality  (Mayer,  2001 ;  Mayer,  Dow,  & 
Mayer,  2003;  Mayer  &  Sims,  1994;  Moreno,  2006). 

Second,  AXL  interactive  modules  build  on  several  multimedia  instructional  design 
principles  thought  to  promote  active  learning.  Open-ended  questions  in  AXL  are  used  to 
encourage  leaders  to  reflect,  not  only  on  various  aspects  of  the  case,  but  on  how  the  case  applies 
to  their  own  personal  experiences.  The  combination  of  personal  experience  and  reflection  helps 
to  capture  the  leader’s  attention,  as  well  as  help  the  leader  to  integrate  new  information  with  past 
knowledge  (Moreno,  2006).  Open-ended  questions  in  AXL  also  are  closely  linked  to  the 
pedagogical  concepts  of  elaborative  interrogation  (e.g.,  Martin  &  Pressley,  1991;  Pressley, 
McDaniel,  Tumure,  Wood,  &  Ahmad,  1987;  Willoughby,  Wood,  Desmarais,  Sims,  &  Kalra, 
1997)  and  self-explanations  (e.g.,  Chi  &  VanLehn,  1991;  Mayer  et  al.,  2003).  During  an  AXL 
module,  leaders  are  provided  with  multiple  discussion  questions  that  prompt  them  to  examine  the 
relationships  among  different  elements  in  the  scenario,  reflect  on  the  causal  factors  that  may 
have  influenced  different  events  in  the  film,  and  consider  the  various  implications  of  different 
courses  of  action  that  could  have  been  adopted  by  characters  in  the  scenario.  By  answering  and 
discussing  these  types  of  questions  (particularly  questions  about  why  certain  events  might  have 
occurred  and  how  group-derived  explanations  correspond  to  personal  experience),  leaders  are 
better  able  to  integrate  new  information  they  encounter  during  the  module  with  previous 
experience  and  knowledge.  These  processes  of  self-explanation  and  elaborative  interrogation 
can  assist  in  retention  of  information  over  time,  as  well  as  generating  inferences  beyond  the 
original  material  contained  in  the  case. 

Like  open-ended  questions,  close-ended  questions  play  an  important  role  in  the  AXL 
system.  While  open-ended  questions  encourage  leaders  to  think  and  actively  construct 
knowledge,  close-ended  questions  allow  leaders  to  receive  feedback,  guidance,  and  more 
targeted  lines  of  questioning.  AXL  uses  close-ended  questions  to  impose  structure  on  an 
otherwise  ill-defined  task  (i.e.,  make  sense  of  a  chaotic  situation)  and  this  imposed  structure  may 
be  particularly  helpful  for  novice  leaders  who  do  not  possess  well-developed  schema  that  would 
help  them  process  the  information  and  issues  embedded  in  a  complex  case  (Moreno,  2006). 

The  opportunity  to  interview  characters  also  builds  on  two  multimedia  instructional 
design  principles:  interactivity  and  control.  The  AXL  capability  to  interview  characters  allows 
leaders  to  test  hypotheses  about  why  characters  performed  particular  actions,  thereby 
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encouraging  “the  processing  of  new  information  by  engaging  students  in  an  active  search  for 
meaning”  (Moreno,  2006,  pg.  65).  Moreover,  the  act  of  formulating  a  question  to  ask  a  character 
requires  leaders  to  process  information  at  a  deep  level;  question-asking  obliges  leaders  to  direct 
their  attention  to  specific  content  and  monitor  their  level  of  comprehension  with  respect  to  that 
content  (Rosenshine,  Meister,  &  Chapman,  1996).  Additionally,  the  character  interview  feature 
offers  a  certain  degree  of  control  to  leaders.  The  discussion  group  must  decide  whether  to 
interview  characters,  what  to  ask  characters,  and  when  to  pose  questions  of  characters.  Allowing 
leaders  to  control  when  and  if  they  want  to  interview  characters  may  help  to  reduce  cognitive 
load,  thus  helping  leaders  to  attend  to  and  process  new  information  more  effectively  (Mayer  et 
ah,  2003). 

A  third  domain  of  research  that  the  AXL  system  draws  from  is  research  on  tacit 
knowledge.  The  work  of  Sternberg  and  colleagues  (e.g.,  Matthew,  Cianciolo,  &  Sternberg, 

2005;  Sternberg,  Forsythe,  Hedlund,  Horvath,  Wagner,  et  ah,  2000)  indicates  that  leaders  can 
acquire  tacit  (i.e.,  actionable)  knowledge  from  reflecting  on  one’s  personal  experiences,  as  well 
as  the  stories  and  experiences  of  others.  During  AXL  instruction,  leaders  have  the  opportunity  to 
reflect  on  multiple  types  of  experiences.  The  first  type  of  experience  that  leaders  are  exposed  to 
in  the  AXL  system  is  the  fictional  situation  depicted  in  the  film,  and  AXL  modules  require 
extensive  reflection  and  analysis  of  the  film.  Leaders  also  examine  a  second  type  of  experience 
in  the  AXL  system — specifically,  leaders  are  asked  to  examine  their  personal  leadership 
experiences.  In  this  instance,  AXL  presents  leaders  with  the  opportunity  to  think  about  their 
personal  experiences  in  a  different  way  than  they  typically  would,  examining  similarities  and 
differences  between  their  personal  experiences  and  the  elements  of  the  case.  Leaders  also  have 
an  opportunity  to  hear  how  peers  respond  to  their  personal  experiences.  A  third  type  of 
experience  that  leaders  are  exposed  to  is  the  personal  experiences  of  other  discussion  group 
members,  and  these  experiences  may  or  may  not  be  similar  to  their  own  personal  experiences. 

The  comparison  between  one’s  personal  experiences  with  the  experiences  of  others  and  the 
situation  depicted  in  the  film  might  be  particularly  beneficial  to  learning  because  research 
suggests  that  comparing  examples  can  assist  in  the  development  of  problem-solving  schema  (see 
Anderson,  Greeno,  Kline,  &  Neves,  1981;  Gick,  1986). 

The  AXL  system  is  still  in  development,  with  a  finalized  system  to  be  completed  in  2007. 
The  earliest  version  of  AXL  included  only  one  film,  Power  Hungry,  and  one  leadership  module. 
Early  research  in  the  AXL  project  focused  on  collecting  leader  reactions  to  the  product  interface 
and  content,  and  the  primary  goal  of  data  collection  was  to  examine  media  effects,  gather 
information  that  would  inform  system  design  choices,  and  refine  the  pedagogical  approach 
(Zbylut  &  Ward,  2004a;  Zbylut  et  al.,  2005).  Based  on  those  results  a  new  version  of  AXL  was 
delivered  in  2006,  with  substantial  changes  to  the  AXL  interface  and  flow  of  instructional  events. 
The  revised  AXL  system  houses  both  the  Tripwire  and  Power  Hungry  films,  as  well  as  several 
new  instructional  modules  for  each  film. 

Before  testing  the  new  AXL  modules,  the  research  team  first  showed  Tripwire  to  small 
groups  of  lieutenants  and  captains  (N=  27)  at  an  Army  installation  in  the  Southwest. 

Researchers  conducted  focus  groups  and  administered  questionnaires  to  the  officers  to  gather 
data  about  reactions  to  the  film  and  to  ascertain  what  leadership  issues  officers  found  to  be 
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salient  in  the  case  study.  Data  from  the  questionnaires  and  focus  group  interviews  were  used  to 
create  measures  to  assess  learning  from  Tripwire  modules. 

ARI  then  pilot  tested  one  of  the  new  AXL  modules  (a  cultural  awareness  module  for 
Tripwire)  at  three  Army  installations.  The  purpose  of  the  investigation  was  to  (a)  collect 
reactions  to  the  module  and  instructional  approach,  (b)  assess  whether  the  module  impacted  how 
leaders  thought  about  the  situation  depicted  in  the  film,  (c)  determine  if  learning  occurred,  and 
(d)  explore  whether  emotional  reactions  were  related  to  learning.  The  next  section  presents  the 
method  and  results  of  that  pilot  investigation. 


PRELIMINARY  EVALUATION  OF  A  CULTURAL  AWARENESS 
MODULE  FOR  TRIPWIRE 

Method 

Participants  and  Procedure 

Participants  were  44  captains  (CPTs),  six  first  lieutenants  (lLTs),  and  five  second 
lieutenants  (2LTs)  from  three  United  States  Army  installations.  Twelve  of  the  officers  had 
deployed  twice  to  the  Middle  East — 1 1  to  Iraq  and  one  to  both  Iraq  and  Afghanistan.  Thirty 
officers  had  deployed  once  to  the  Middle  East — 26  to  Iraq  and  four  to  Afghanistan.  The 
remaining  13  officers  had  not  deployed  to  either  Iraq  or  Afghanistan. 

Each  officer  participated  in  a  small  discussion  group  for  the  cultural  awareness  module  of 
Tripwire.  Discussion  group  sizes  tended  to  range  from  two  to  four  junior  officers,  although  one 
training  session  had  one  officer,  and  another  training  session  contained  seven  officers.1 

Prior  to  watching  the  film,  officers  completed  a  brief  questionnaire  asking  them  to  report 
on  their  deployment  experience  and  level  of  emotional  arousal.  Officers  then  watched  Tripwire 
as  a  group  on  a  laptop  computer.  After  watching  the  film,  officers  independently  completed  a 
second  set  of  measures.  Once  all  of  the  officers  completed  the  second  set  of  measures,  they 
began  the  discussion  phase  of  the  cultural  awareness  module.  The  module  consisted  of  a  series 
of  open-ended  and  close-ended  questions  to  prompt  discussion  about  the  cultural  issues  in 
Tripwire ,  particularly  concerning  the  treatment  of  locals,  managing  the  perceptions  of  locals,  and 
dealing  with  IEDs  in  an  urban  environment.  Discussion  lasted  approximately  45  to  90  minutes. 
Upon  completing  the  cultural  awareness  module,  officers  independently  completed  a  final  set  of 
measures.  The  entire  process,  including  completion  of  measures,  ranged  from  90  to  120  minutes. 


1  Eight  officers  were  originally  scheduled  to  participate  in  two  discussion  groups  consisting  of  four  officers  each,  but 
three  of  the  officers  attended  the  wrong  discussion  group.  This  resulted  in  seven  officers  in  one  group,  and  one 
officer  in  another  “group.”  The  officer  who  completed  the  cultural  awareness  module  by  himself  was  told  to  reflect 
on  the  questions  contained  in  the  module.  This  individual  had  been  deployed  previously  and  presumably  could  draw 
from  his  experience  while  reflecting  on  the  scenario. 
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Measures 


Demographic  Items 

Prior  to  training,  officers  indicated  their  rank,  how  many  times  they  had  been  deployed  to 
Iraq,  and  how  many  times  they  had  been  deployed  to  Afghanistan. 

Emotional  Measures 

Self-reported  emotional  arousal  Officers  completed  five  self-report  items  that  gauged 
their  level  of  emotional  arousal.  These  five  items  were  part  of  Mehrabian  and  Russell’s  (1974) 
arousal  scale  and  were  anchored  with  bipolar  adjectives  on  a  nine-point  scale  ranging  from  -4  to 
+4.  The  bipolar  anchors  were  relaxed/stimulated,  calm/excited,  sluggish/neutral,  dull/jittery,  and 
sleepy/wide  awake.  Arousal  was  operationalized  as  the  mean  of  the  five  items.  The  arousal 
measure  was  given  at  three  time  points:  immediately  prior  to  watching  Tripwire  (a  =  .66), 
immediately  after  watching  Tripwire  (a  =  .81),  and  immediately  after  completing  the  cultural 
awareness  training  module  (a  =  .74). 

Positive  and  negative  affect.  After  watching  the  film  and  completing  a  measure  of 
emotional  arousal,  officers  completed  the  Positive  and  Negative  Affect  Schedule  (PANAS; 
Watson,  Clark,  &  Tellegen,  1988).  Officers  indicated  the  extent  to  which  they  experienced  20 
emotions  while  watching  the  film.  The  20  items  were  anchored  on  a  five-point  scale  ranging 
from  “very  slightly  or  not  at  all”  (1)  to  “extremely”  (5).  Positive  affect  (a  =  .91)  was  the  mean  of 
10  positive  emotions,  such  as  active,  excited,  and  proud.  Negative  affect  (a  =  .85)  was  the  mean 
of  10  negative  emotions,  such  as  hostile,  upset,  and  distressed. 

Reactions  to  the  Film 

After  completing  the  PANAS,  officers  completed  15  items  that  addressed  reactions  that 
they  had  to  the  film.  All  of  the  items  were  anchored  on  a  seven-point  scale  ranging  from 
“strongly  disagree”  (1)  to  “strongly  agree”  (7).  These  items  were  completed  prior  to  participating 
in  discussion. 

Character  depth.  Character  depth  was  operationalized  as  the  mean  of  six  items  that 
assessed  perceptions  about  how  believable  and  complex  the  characters  in  the  film  were  (a  =  .78). 
Examples  of  items  from  the  scale  were,  “The  story  did  a  good  job  of  communicating  the 
thoughts,  feelings,  and  emotions  of  the  characters,”  “I  can  imagine  meeting  people  similar  to  the 
characters  presented  in  the  story,”  and  “I  could  identify  with  at  least  one  character  in  the  story.” 

Film  involvement.  Officers  answered  four  items  that  assessed  the  degree  to  which  they 
felt  involved  in  the  film’s  storyline.  These  items  asked  the  extent  to  which  officers  wanted  to 
find  out  what  happened  next  in  the  story,  whether  the  story  was  involving,  and  whether  the  story 
was  suspenseful.  An  additional  item,  which  was  reverse-scored,  asked  officers  if  they  were 
bored  while  watching  the  movie.  Film  involvement  was  operationalized  as  the  mean  of  the  four 
items  (a  =  .79). 
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Confusion.  Confusion  was  assessed  using  the  item,  “I  was  confused  about  what  was 
going  on  in  the  story.” 

Realism.  Officers  rated  the  extent  to  which  they  believed  the  story  was  realistic  by  using 
the  item,  “The  story  was  realistic.” 

Preference  for  film  over  traditional  instructional  methods.  Officers  completed  two 
items  that  assessed  their  preference  for  filmed  case  studies  over  traditional  methods  of 
instruction  found  in  the  Army.  The  first  item  was,  “In  an  educational  context,  I  would  rather 
watch  a  film  like  this  than  read  a  scenario.”  The  second  item  was,  “In  an  educational  context,  I 
would  prefer  to  discuss  a  film  like  this  than  listen  to  a  PowerPoint  briefing.” 

Reactions  to  AXL  Training 

After  finishing  the  discussion  phase  of  the  cultural  awareness  module  and  the  final 
measure  of  arousal,  officers  completed  16  items  that  addressed  various  reactions  that  they  had  to 
the  AXL  system.  Items  were  anchored  on  a  seven-point  scale  ranging  from  “strongly  disagree” 
(1)  to  “strongly  agree”  (7). 

Global  reactions  to  training.  Officers  answered  five  items  that  assessed  the  perceived 
value  of  the  AXL  tool,  ease  of  use,  how  informative  the  training  was,  how  responsive  the 
training  was  to  the  officer’s  educational  needs,  and  how  frustrating  the  training  approach  was 
(reverse-scored).  A  score  for  global  reactions  to  the  training  was  formed  by  computing  the  mean 
of  the  five  items  (a  =  .80). 

Stimulated  thought.  Because  one  of  the  objectives  of  case  method  teaching  is  to 
encourage  critical  thinking,  four  items  assessed  whether  officers  believed  that  the  cultural 
awareness  module  made  them  think.  These  items  were  “The  questions  included  as  part  of  the 
training  made  me  think  critically  about  issues  in  the  scenario,”  “This  exercise  made  me  think 
about  what  makes  a  leader  effective,”  “This  exercise  made  me  think  about  what  makes  a  leader 
ineffective,”  and  “The  training  made  me  think  about  the  scenario  in  a  different  way  than  I 
usually  would.”  A  score  for  this  scale  was  calculated  by  computing  the  mean  of  the  four  items 
(a  =.75). 


Character  interactivity.  Officers  completed  four  items  that  addressed  their  reactions  to 
the  character  interview  features  of  the  AXL  system.  These  items  were  “Interviews  with  the 
characters  gave  me  insight  into  why  the  mission  failed,”  “I  felt  like  the  characters  answered  the 
questions  that  they  were  asked,”  “Interviews  with  the  characters  helped  me  understand  why  there 
was  a  breakdown  in  communication  between  different  characters,”  and  “I  liked  that  the  actors 
stayed  ‘in  character’  when  they  answered  my  questions.”  A  score  for  this  scale  was  calculated 
by  computing  the  mean  of  the  four  items  (a  =  .81). 

Applicability  of  training.  Officers  completed  a  single  item,  “I  can  apply  some  of  the 
things  that  I  learned  here  to  my  activities  as  a  leader,”  which  was  used  to  assess  whether  officers 
believed  they  would  transfer  knowledge  gained  during  the  AXL  module  to  their  job. 
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Wanted  feedback.  In  order  to  assess  whether  officers  believed  the  AXL  system  should 
provide  more  feedback,  officers  indicated  the  extent  to  which  they  agreed  with  the  following 
item:  “I  wanted  more  feedback  about  how  I  was  doing  during  the  training  modules.” 

Learning 

One  challenge  to  assessing  whether  learning  occurred  during  AXL  instruction  is  that  the 
AXL  case  method  approach  adopts  a  constructivist  view  of  learning.  According  to  the 
constructivist  paradigm,  meaningful  learning  occurs  when  individuals  construct  their  own 
interpretations  of  the  world.  Moreover,  multiple  valid  interpretations  of  the  world  are  not  only 
possible,  but  expected  (Dalgamo,  2001;  Hannafin  et  al.,  1997;  Mayer,  1996).  In  this  respect, 
discussing  a  case  study  is  largely  a  sensemaking  process  in  which  groups  of  leaders  share 
experiences  and  thoughts  to  derive  a  greater  understanding  about  leadership  than  they  had  prior 
to  discussion.  Given  that  the  content  and  outcomes  of  different  discussion  groups  may  have 
significant  variability  across  groups,  a  primary  challenge  for  the  researcher  is  how  to  objectively 
measure  whether  discussion  was  worthwhile.  That  is,  if  different  leaders  are  learning  different, 
but  valid  things,  how  does  one  measure  learning? 

While  leaders  from  any  one  discussion  group  may  discuss  different  information  and 
ultimately  arrive  at  different  “wisdom”  than  leaders  from  another  discussion  group,  the  tenets  of 
constructivism  clearly  imply  that  individuals  who  engage  in  knowledge  construction  activities 
(like  discussion)  should  experience  a  change  in  their  understanding  or  interpretation  of  events. 
Additionally,  if  the  researcher  has  a  sense  of  what  common  themes  or  topics  will  emerge  in 
different  discussion  groups,  the  researcher  can  compile  a  heterogeneous  set  of  learning  items  that 
will  likely  tap  at  least  a  few  of  the  topics  brought  up  during  any  one  discussion. 

Learning  in  this  investigation  was  assessed  using  two  different  approaches.  The  first 
approach  utilized  a  rank-ordering  task  to  assess  whether  leaders  experienced  a  change  in  their 
understanding  of  the  importance  of  cultural  issues  embedded  in  the  case  study.  The  second 
approach  used  a  series  of  forced-choice  items  to  examine  whether  leaders  demonstrated  better 
judgment  after  completing  the  cultural  awareness  module.  While  the  first  approach  provided  an 
indicator  of  how  a  leader  was  framing  issues  and  content  embedded  in  Tripwire,  the  second 
approach  provided  an  indicator  that  the  leader  applied  knowledge  learned  during  discussion  to 
problems  embedded  in  Tripwire. 

Emphasis  on  cultural  issues.  As  mentioned  previously,  an  earlier  data  collection  effort 
used  focus  groups  and  written  questionnaires  to  explore  the  leadership  issues  junior  officers 
believed  were  important  in  Tripwire.  During  that  investigation,  officers  indicated  what  they 
would  have  done  differently  if  they  were  a  Soldier  in  the  Tripwire  scenario.  From  this  data,  a  list 
of  21  issues  was  compiled.2 


2  The  initial  list  consisted  of  92  character  actions  discussed  or  written  about  during  focus  group  sessions.  Actions 
having  significant  overlap  were  combined,  resulting  in  a  list  of  47  items.  Because  rank  ordering  47  issues  would  be 
cognitively  overwhelming  for  participants,  the  list  was  reduced  to  the  28  most  commonly  mentioned  issues.  A  rank¬ 
ordering  task  with  these  28  items  was  piloted  at  one  of  the  three  Army  installations  during  data  collection.  Analysis 
of  the  first  administration  of  the  measure  indicated  that  some  issues  were  selected  infrequently,  and  seven  issues 
were  removed  from  the  measure  to  make  the  ranking  task  less  cognitively  demanding.  Thus,  the  finalized  ranking 
task  of  21  items  was  administered  at  only  two  of  the  three  installations,  resulting  in  n  —  36  on  this  measure. 
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Eight  of  the  21  issues  appeared  as  discussion  topics  in  the  cultural  awareness  module. 
These  issues  not  only  reflected  decisions  and  actions  that  could  influence  Iraqi  behavior  and 
perceptions  of  the  Army,  but  also  included  issues  that  required  knowledge  about  cultural 
antecedents  and  consequences  surrounding  events  in  the  scenario.  Cultural  issues  included 
somewhat  evident  cultural  topics  such  as,  “CPT  Holston  should  have  refrained  from  shooting  the 
backpack  with  the  Koran”  and  “1LT  Porter  should  have  refrained  from  striking  the  second 
interpreter,”  as  well  as  less  apparent  cultural  topics  such  as,  “CPT  Holston  should  have 
investigated  why  the  first  interpreter  was  shot.” 

The  21  issues  were  used  in  a  rank-ordering  task  designed  to  measure  a  leader’s 
understanding  of  what  issues  he  or  she  believed  were  most  important  in  the  film.  In  the  rank¬ 
ordering  task,  officers  were  asked  to  select  the  top  seven  actions  that  Soldiers  in  the  film  should 
have  done  differently.  Officers  were  then  asked  to  arrange  those  top  seven  actions  from  most 
important  to  least  important.  Officers  received  one  point  for  each  cultural  issue  that  they  ranked 
among  their  top  seven  most  important  issues. 

Officers  completed  the  rank  ordering  task  twice — once  after  watching  the  film  (but 
before  discussion)  and  once  after  discussion.  The  21  issues  were  arranged  in  different  orders  for 
the  two  time  points.  The  ranking  task  was  used  to  examine  whether  officers  deemed  the  cultural 
issues  to  be  more  important  after  discussing  the  scenario. 

Behavioral  judgment  In  addition  to  the  ranking  task,  leaders  completed  eight  items  that 
asked  them  to  choose  between  two  courses  of  action  that  could  have  been  taken  during  the 
Tripwire  scenario.  The  forced-choice  items  represented  a  heterogeneous  set  of  issues 
intentionally  embedded  in  the  case  study,  including  maintaining  one’s  health  versus  self-sacrifice 
for  the  mission,  accomplishing  the  mission  versus  protecting  one’s  Soldiers,  listening  to  the 
advice  of  Soldiers  versus  maintaining  command  authority,  and  neutralizing  potential  IED  threats 
versus  offending  Iraqi  civilians.  For  each  item,  a  leader  received  one  point  if  he  or  she  selected 
the  better  course  of  action  (i.e.,  consistent  with  the  teaching  goals  built  into  Tripwire).  A 
behavioral  judgment  score  was  then  computed  by  summing  across  the  eight  items.  Items  and 
response  options  were  randomly  arranged  for  both  the  pretest  and  posttest.  Additionally,  one  of 
the  “pretest”  items  was  given  prior  to  watching  the  film  because  it  did  not  require  watching  the 
film  to  answer  the  question.  Two  examples  of  forced-choice  items  are  provided.  The  “correct” 
answers  are  denoted  by  an  asterisk. 

During  the  planning  meeting,  the  platoon  leaders  told  CPT  Holston  that  they  should  be 
conducting  “rolling  Ts”  in  the  town  in  order  to  keep  things  calm  until  the  VIP’s 
arrival.  After  the  first  interpreter  was  killed,  CPT  Holston’s  Soldiers  again  indicated 
that  they  believed  a  more  aggressive  stance  should  be  taken  in  the  town.  Do  you 
believe  that  it  is  more  important  for  CPT  Holston  to. . . 

a.  Maintain  his  initial  restrained  stance,  especially  given  that  his  junior  officers 
are  challenging  him. 

b.  Act  on  the  advice  of  his  subordinates,  even  if  they  are  challenging  his 
authority.* 
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In  general,  it  is  better  for  a  leader  to. . . 

a.  Put  mission  tasks  ahead  of  physical  well-being  (e.g.,  skipping  meals,  reducing 
sleep  time)  in  order  to  get  the  job  done. 

b.  Take  time  out  to  sleep,  eat,  and  hydrate,  even  if  it  takes  some  time  away  from 
important  mission-related  tasks.* 


Results 

Participant  Reactions 

The  means  and  standard  deviations  for  the  measures  used  in  this  investigation  are 
presented  in  Table  1 .  Results  indicated  that  reactions  to  the  film  were  positive,  with  officers 
rating  the  film  as  involving  (M=  5.55,  SD  =  1.03)  and  characters  as  complex  (M=  5.35,  SD  = 
.93).  In  general,  officers  tended  to  view  the  film  as  realistic,  although  substantial  variability 
existed  among  leaders  (M  =  4.96,  SD  —  1.71).  Regardless  of  whether  the  film  was  perceived  as 
realistic,  officers  overwhelmingly  preferred  watching  a  film  to  reading  a  scenario  (M  =  6.60,  SD 
=  .81)  or  listening  to  a  PowerPoint  presentation  (M=  6.76,  SD  =  .58). 

Reactions  to  the  AXL  training  package  overall  also  tended  to  be  positive.  Officers’ 
global  reactions  to  the  training  were  positive  ( M=  5.64,  SD  =  .85),  and  officers  indicated  that  the 
cultural  awareness  module  was  thought  provoking  (M=  5.39,  SD  =  .82).  Likewise,  officers 
indicated  they  could  apply  something  that  they  learned  during  the  module  to  their  activities  as  a 
leader  (M  =  5.56,  SD  =  1.18).  Officers  were  somewhat  neutral  with  respect  to  whether  they 
believed  the  AXL  system  should  provide  more  feedback  ( M  =  3.45,  SD  =  1.71),  with  some 
officers  wanting  substantially  more  feedback  than  others.  The  character  interview  features 
received  slightly  positive  ratings  (M=  4.53,  SD  =  1 .23),  but  ratings  suggest  that  the  character 
interview  functionality  in  the  system  needs  improvement. 

Learning 


Emphasis  on  Cultural  Issues 

A  paired-samples  /-test  was  conducted  to  examine  if  officers  placed  stronger  emphasis  on 
cultural  issues  after  completing  the  cultural  awareness  module.  Results  indicated  that  officers 
ranked  more  cultural  issues  as  important  after  participating  in  discussion  (M=  3.11  issues,  SD  = 
1.39)  than  before  discussion  (M  =  2.64  issues,  SD  =  1.17),  /(35)  =  2.50,  p  <  .05.  This  finding 
provides  evidence  that  the  cultural  awareness  module  resulted  in  a  conceptual  shift  in  officer 
understanding  of  the  situation  depicted  in  the  Tripwire  scenario,  such  that  officers  viewed 
cultural  issues  as  significantly  more  important  after  completing  the  cultural  awareness  module. 
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Table  1 

Means  and  Standard  Deviations  of  Variables  (N  =  55) 


M 

SD 

Reactions  to  the  Film 

Character  Depth 

5.35 

.93 

Involving  Film 

5.55 

1.03 

Confused 

1.78 

.92 

Realism 

4.96 

1.71 

Preference  for  Film  over  Text  Scenario 

6.60 

.81 

Preference  for  Film  over  PowerPoint 

6.76 

.58 

Reactions  to  Training 

Global  Training  Reactions 

5.64 

.84 

Stimulated  Thought 

5.39 

.82 

Character  Interactivity 

4.53 

1.23 

Training  Applicability 

5.56 

1.18 

Wanted  Feedback 

3.45 

1.71 

Learning 

Behavioral  Judgment  Pretest 

4.82 

1.22 

Behavioral  Judgment  Posttest 

5.08 

1.30 

Emphasized  Cultural  Issues  (Post-film) 3 

2.64 

1.17 

Emphasized  Cultural  Issues  (Post-training) a 

3.11 

1.39 

Emotional  Measures 

Arousal  (Pre-film) 

-.47 

1.13 

Arousal  (Post-film) 

.80 

1.30 

Arousal  (Post-training) 

.20 

1.10 

Positive  Affect 

3.03 

.85 

Negative  Affect 

1.82 

.67 

Note .  Measures  included  in  the  Reactions  to  Film  and  Reactions  to  Training  categories  are  based  on  a  7-point  scale 
ranging  from  “strongly  disagree”  to  “strongly  agree.”  Within  the  Learning  category,  the  highest  possible  score  that 
could  be  achieved  was  8  for  the  behavioral  judgment  test  and  7  for  the  Emphasis  on  Cultural  Issues  task.  With 
respect  to  the  Emotional  Measures  category,  self-reports  of  arousal  were  on  scale  ranging  from  -4  to  +4,  with 
positive  scores  indicating  arousal  and  negative  scores  indicating  lethargy.  Positive  and  negative  affect  were 
measured  on  a  5-point  scale  with  higher  scores  indicating  stronger  affect. 

a  «  =  36 
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Changes  in  Behavioral  Judgment 

A  paired-samples  /-test  also  was  used  to  examine  whether  discussing  the  film  resulted  in 
better  judgment  by  leaders.  Results  indicated  that  officers  tended  to  exhibit  better  judgment 
about  behavioral  courses  of  action  after  discussion  {M-  5.08,  SD  =  1.30)  than  they  did  before 
discussion  (M=  4.82,  SD  =  1.22),  /(54)  =  2.0$, p  <  .05.  Thus,  the  cultural  awareness  module 
appears  to  have  impacted  leaders’  judgment  in  a  way  consistent  with  the  teaching  goals  of  the 
Tripwire  case  study. 


Emotional  Response  to  the  Training 

A  repeated  measures  analysis  of  variance  (ANOVA)  indicated  that  self-reported  levels  of 
arousal  changed  at  different  points  during  the  cultural  training,  F( 2,  108)  =  25.99 ,p  <  .001. 
Paired-samples  /-tests  were  conducted  to  compare  the  three  time  points  to  examine  the  nature  of 
these  differences.  Results  indicated  that  arousal  was  lowest  prior  to  watching  the  film  (M=  -.47, 
SD  -  1.13),  being  significantly  lower  than  arousal  levels  reported  immediately  after  the  film 
(M=  .80,  SD  =  1.30),  /(54)  =  -6.33,  p<  .001,  and  after  the  training  (M=  .20,  SD  =  1.10), 

/(54)  =  -4.04 ,p  <  .001 .  In  general,  arousal  was  highest  immediately  after  watching  the  film,  with 
arousal  weakening  slightly  during  the  course  of  discussion,  /(54)  =  2.16,  p  <  .001.  The  pattern  of 
arousal  throughout  the  training  is  depicted  in  Figure  5. 


Figure  5.  Reported  Arousal  over  the  Course  of  the  Training  Intervention 


These  results  suggest  that  the  training  is  emotionally  engaging,  with  the  film  being 
particularly  arousing.  However,  while  the  arousal  measure  provides  an  indicator  of  the 
energizing  component  of  emotion,  it  does  not  provide  an  indicator  of  the  valence  (i.e., 
pleasantness)  of  emotion.  Affect  consists  of  two  important  elements:  level  of  activation  (i.e., 
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arousal)  and  hedonic  relevance  (Feldman,  1995;  Russell,  2003).  With  respect  to  Tripwire,  it  was 
unclear  whether  the  film  would  be  received  as  a  pleasant  or  an  unpleasant  experience.  In  one 
respect,  the  film  could  generate  positive  affect,  because  the  film  might  prompt  the  interest, 
attention,  and  excitement  of  the  viewer.  Conversely,  the  unpleasant  situation  depicted  in  the  film 
could  prompt  a  negative  experience,  generating  feelings  of  anger,  hostility,  and  distress.  A 
paired-samples  t-test  was  conducted  to  examine  whether  the  film  evoked  more  positive  versus 
negative  affect.  The  Mest  indicated  that  the  film  evoked  greater  levels  of  positive  affect  (M= 
3.03,  SD  =  .85)  than  negative  affect  (M=  1.82,  SD  =  .67),  t( 54)  =  1 1.62 ,p<  .001. 

Relationships  between  Emotions  and  Learning 

One  of  the  purposes  of  this  investigation  was  to  explore  the  relationship  between 
emotions  and  learning.  Of  note  is  how  emotional  measures  correlated  with  other  measures  (see 
Table  2).  While  arousal  levels  were  not  significantly  related  to  any  learning  variables,  affect  was 
related  to  several  learning-relevant  variables.  Positive  affect  was  positively  correlated  with  the 
behavioral  judgment  posttest  score  ( r  =  .33,  p  <  .05)  but  was  uncorrelated  with  the  behavioral 
judgment  pretest  score  (r  =  .  1 0,  p  =  ns),  suggesting  that  positive  affect  experienced  during  the 
film  may  be  related  in  some  way  to  what  an  individual  learns  during  the  course  of  discussion. 
Positive  affect  also  was  related  to  global  reactions  to  the  training  (r  =  A3,p  —  .001),  and  both 
positive  affect  and  negative  affect  were  related  to  how  thought-provoking  the  training  was 
(positive  affect:  r  =  .3 1  ,P<  .05;  negative  affect:  r  =  .44,/?  =  .001),  and  whether  leaders  believed 
they  learned  anything  that  could  be  applied  to  their  work  (positive  affect:  r  =  .43,/?  =  .001; 
negative  affect:  r  =  .3 1 ,  p  <  .05). 

Results  with  respect  to  the  relationship  between  affect  and  emphasis  on  cultural  issues 
were  more  difficult  to  interpret.  Neither  positive  affect  (r  =  .26,/?  =  ns)  or  negative  affect  (r  = 
.21,/?  =  ns)  were  significantly  related  to  emphasis  on  cultural  issues  after  the  discussion. 
However,  positive  affect  was  significantly  correlated  with  how  much  officers  emphasized 
cultural  issues  prior  to  discussion  (r  =  .44,/?  <  .01).  Thus,  officers  who  experienced  more 
positive  affect  during  the  film  appeared  to  focus  their  attention  on  culturally  relevant  courses  of 
action  before  discussion  with  other  officers,  but  the  relationship  between  positive  affect  and 
emphasis  on  cultural  issues  was  weaker  after  participating  in  discussion. 

Several  other  correlations  are  worth  noting.  First,  an  officer’s  level  of  engagement  with 
the  film  might  have  bearing  on  how  well  that  individual  learns  from  the  training.  Officers  who 
found  the  characters  to  be  complex  and  believable  tended  to  score  higher  on  the  judgment 
posttest  than  officers  who  did  not  rate  the  film  characters  highly  ( r  -  .30,/?  <  .05).  Additionally, 
officers  who  rated  the  film  as  involving  tended  to  score  higher  on  the  judgment  posttest  than 
officers  who  did  not  rate  the  film  as  involving  (r  =  .35,  p  <  .01). 

Second,  unanticipated  relationships  were  found  between  pre-film  arousal  levels  and 
reactions  to  both  the  film  and  training.  Officers  who  reported  higher  levels  of  arousal  prior  to 
watching  the  film  also  reported  higher  levels  of  confusion  about  what  was  happening  in  the  film 
(r  =  .29,  p  <  .05)  and  wanted  to  receive  more  feedback  during  training  (r  =  .30,/?  <  .05). 

Officers  with  higher  levels  of  arousal  prior  to  watching  the  film  also  tended  to  indicate  that  they 
did  not  prefer  film  to  a  PowerPoint  briefing  (r  =  -.36,  p  <  .01). 
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Table  2 

_ Correlations  among  Measures  (N  -  55) 
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Table  2  (continued) 

Correlations  among  Measures  (N  =  55) 
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DISCUSSION 


The  results  of  this  investigation  indicated  that  the  case  study  approach  adopted  in  the 
AXL  system  holds  promise  in  shaping  the  knowledge  and  judgment  of  Army  leaders. 
Specifically,  after  completing  the  cultural  awareness  module,  leaders  exhibited  better  judgment 
about  various  behavioral  courses  of  action  that  could  have  been  adopted  in  the  Tripwire  scenario. 
The  cultural  awareness  module  also  appeared  to  play  a  role  in  influencing  the  importance  leaders 
placed  on  the  cultural  issues  in  the  film,  with  leaders  placing  stronger  emphasis  on  cultural  issues 
after  having  completed  the  module.  These  findings  are  consistent  with  research  on  tacit 
knowledge  that  suggests  individuals  can  develop  tacit  knowledge  by  reflecting  on  their 
experiences  and  the  experiences  of  others  (Cianciolo,  Anotonakis,  &  Sternberg,  2004;  Matthew 
et  ah,  2005;  Sternberg  et  ah,  2000). 

Findings  with  respect  to  reactions  to  the  Tripwire  film  were  both  encouraging  and  similar 
to  findings  for  the  Power  Hungry  film  (Zbylut  &  Ward,  2004a;  Zbylut  et  ah,  2005).  Officers 
rated  Tripwire  as  involving  and  the  characters  as  multi-layered.  Moreover,  officers  found  the 
film  to  be  fairly  realistic  and  were  not  confused  by  the  film.  Indeed,  officers  overwhelmingly 
indicated  that  they  would  prefer  to  watch  a  film  rather  than  read  a  case  study  or  listen  to  a 
PowerPoint  presentation,  both  of  which  are  common  instructional  techniques  used  in  Army 
education.  These  results  indicate  that  the  two  films  are  an  appropriate  way  for  depicting  case 
study  material  in  the  AXL  system. 

In  addition  to  positive  reactions  to  the  film,  officers  responded  positively  to  the  AXL 
cultural  awareness  module  as  a  whole.  Officers  rated  the  AXL  cultural  module  as  valuable  and 
useful.  More  importantly,  officers  indicated  that  they  would  be  able  to  transfer  something  that 
they  learned  to  their  activities  as  a  leader.  Furthermore,  officers  reported  that  the  module  was 
thought-provoking — not  only  compelling  them  think  about  what  makes  a  leader  effective  versus 
ineffective,  but  helping  them  to  think  about  the  Tripwire  scenario  in  a  different  way  than  they 
usually  would.  Taken  as  a  whole,  the  results  of  this  investigation  support  some  of  the  basic 
tenets  of  case  method  teaching;  namely,  case  method  teaching  challenges  the  assumptions  of 
students  and  stimulates  critical  thinking  (e.g.,  Crittenden  et  al.,  1999;  Jennings,  1996;  Kreber, 
2001;  Richardson,  1994;  Stewart  &  Dougherty,  1993). 

This  investigation  also  explored  the  relationship  between  emotions  and  learning.  One  of 
the  long-term  goals  of  the  AXL  project  is  to  examine  if  film  is  emotionally  evocative  and 
whether  this  has  any  long  term  benefits  for  learning  and  behavior  over  time.  Previous  research 
regarding  AXL  indicated  that  the  first  film,  Power  Hungry,  led  to  heightened  emotional  arousal 
and  reports  of  positive  affect  (Zbylut  &  Ward,  2004a;  Zbylut  et  ah,  2005),  and  results  found  for 
Tripwire  replicate  those  findings.  While  the  present  investigation  did  not  examine  the  long-term 
impact  of  emotionality  on  learning  and  retention,  results  indicated  a  positive  relationship 
between  positive  affect  and  behavioral  judgment  posttest  scores.  Moreover,  both  positive  affect 
and  negative  affect  were  related  to  several  self-report  measures  indicative  of  learning — 
specifically,  how  much  leaders  reported  that  the  training  stimulated  thought  and  the  extent  to 
which  leaders  could  transfer  knowledge  to  the  work  environment.  Due  to  the  design  of  this 
research,  it  is  unclear  whether  affect  played  a  role  in  learning  or  was  just  a  byproduct  of  the  film 
medium.  However,  a  growing  body  of  work  on  mood,  affect,  and  arousal  support  the  notion  that 
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emotions  play  an  integral  role  in  learning.  For  example,  Murray,  Harish,  Hirt,  and  Sujan  (1990) 
found  that  individuals  in  positive  moods  demonstrated  enhanced  cognitive  flexibility  and  had 
greater  access  to  diverse  and  unusual  information  than  individuals  in  neutral  moods.  Both 
cognitive  flexibility  (i.e.,  being  able  to  identify  similarities  and  differences  between  stimuli)  and 
access  to  novel  information  would  likely  enhance  case  method  discussion  because  they  increase 
the  probability  that  leaders  will  (1)  bring  unique  perspectives  to  group  discussion,  and  (2)  be  able 
to  compare,  contrast,  and  ultimately  integrate  those  differing  perspectives,  resulting  in  new 
knowledge.  Like  positive  affect,  negative  affect  might  play  a  productive  role  in  learning  from 
case  studies  because  negative  moods  may  trigger  a  desire  to  process  information  more  deeply 
and  to  think  more  analytically  (George,  2000;  Sinclair,  Mark,  &  Clore,  1994).  The  results  of  the 
present  investigation  are  consistent  with  the  proposition  that  negative  affect  is  related  to  thinking 
in  that  officers  who  reported  experiencing  negative  affect  during  the  film  also  tended  to  report 
that  the  cultural  awareness  module  was  thought-provoking.  However,  negative  affect  did  not 
significantly  correlate  with  the  behavioral  judgment  posttest,  which  is  another  indicator  of 
critical  thinking. 

It  is  worth  noting  that  the  measure  of  positive  affect  used  in  this  investigation  included 
several  items  that  measured  emotions  that  could  be  considered  conducive  to  the  learning  process 
(e.g.,  alert,  attentive,  enthusiastic,  and  interested).  One  explanation  for  why  learning  measures 
and  positive  affect  were  correlated  is  that  leaders  who  experienced  positive  affect  might  have 
paid  better  attention  to  the  details  of  the  film,  and  leaders  were  then  able  to  bring  more  film 
content  into  discussion  and  analysis  during  their  discussion  groups.  The  unexpected  correlation 
between  positive  affect  and  emphasis  on  cultural  issues  prior  to  discussion  of  the  film  may  serve 
as  evidence  to  support  that  claim.  Prior  to  watching  the  film,  leaders  were  informed  that  they 
would  be  completing  a  module  on  cultural  awareness,  so  it  is  possible  that  leaders  who  were 
more  alert  and  attentive  tended  to  focus  more  on  the  cultural  issues  in  the  film.  As  a  result, 
individuals  who  experienced  greater  positive  affect  also  tended  to  place  greater  weight  on 
cultural  content  during  the  ranking  task.  Unfortunately,  the  data  collection  design  used  in  this 
research  does  not  merit  such  causal  interpretations,  and  the  correlation  between  positive  affect 
and  emphasis  on  cultural  issues  is  based  on  a  relatively  small  sample  size. 

Another  potential  explanation  for  the  relationship  between  positive  affect  and  learning 
could  be  that  leaders  carry  over  the  positive  affect  that  they  experienced  during  the  film  and  then 
brought  that  energy  and  enthusiasm  to  the  discussion.  Further  research  is  necessary  to  explore 
the  potential  mechanisms  by  which  positive  affect,  negative  affect,  and  other  emotional  variables 
may  benefit  or  harm  the  learning  process,  particularly  in  the  context  of  group  discussion.  One 
line  of  inquiry  that  might  stimulate  research  on  how  emotions  impact  group  discussion  is  the 
work  on  emotional  contagion,  which  suggests  that  individuals  can  infect  others  with  their  mood 
(Barsade,  2002;  Gump  &  Kulik,  1997;  Howard  &  Gengler,  2001;  Pugh,  2001;  Sy,  Cote,  & 
Saavedra,  2005). 

In  sum,  the  findings  of  this  research  are  especially  encouraging  given  that  many  of  the 
leaders  who  participated  have  significant  deployment  experience,  and  yet  still  appeared  to  learn 
something  new  from  the  AXL  cultural  awareness  module.  Moreover,  the  AXL  system  delivers 
an  online  format  for  conducting  effective  case  method  instruction  without  requiring  intervention 
from  an  instructor.  Such  a  format  reduces  demands  on  instructor  time  in  the  form  of  lesson 
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preparation  and  helps  to  deliver  interesting  educational  content  in  an  online  learning 
environment.  The  AXL  system,  however,  is  flexible  with  respect  to  allowing  instructors  to  be  as 
involved  in  instruction  as  they  wish  to  be.  While  instructor  involvement  is  not  required  to 
administer  AXL  modules,  instructors  can  use  AXL  films  to  conduct  their  own  class  discussions 
around  whatever  topics  they  choose  (Zbylut  &  Ward,  2004a,  2004b).  Additionally,  the  finalized 
AXL  system  will  allow  instructors  to  create  their  own  online  modules  around  either  of  the  two 
existing  films  or  upload  new  text-based  or  filmed  case  studies  into  the  AXL  system’s  library. 

Challenges  in  Evaluating  AXL  Research 

Research  on  the  AXL  system  is  ongoing,  with  some  of  the  most  important  evaluation 
events  to  occur  in  a  longitudinal  investigation  scheduled  for  2007.  Although  the  results  of  this 
pilot  investigation  were  encouraging,  this  research  examined  only  one  of  the  several  AXL 
modules  currently  available.  While  findings  provided  evidence  that  the  cultural  awareness 
module  is  a  useful  instructional  tool,  the  AXL  intervention  as  a  whole  consists  of  two  case 
studies  and  multiple  leadership  modules.  Our  expectation  is  that  the  most  meaningful  learning 
occurs,  not  from  a  single  module,  but  from  discussion  of  multiple  topics  across  the  different 
situations  depicted  in  the  two  filmed  case  studies.  Moreover,  it  is  unclear  from  this  investigation 
how  long  the  effects  in  learning  will  last  over  time,  or  whether  leader  behavior  and  judgment  has 
pennanently  changed  as  a  result  of  the  intervention. 

The  biggest  obstacle  to  evaluating  AXL  instruction  has  been  access  to  sufficient  numbers 
of  Army  officers  for  sufficient  periods  of  time.  Given  the  frequency  of  deployments  to  both  Iraq 
and  Afghanistan,  the  typical  pool  of  leaders  available  to  participate  in  research  is  greatly 
reduced.  Moreover,  when  leaders  are  stateside,  they  are  busy  preparing  their  units  for  the  next 
deployment  and  do  not  have  time  to  participate  in  research,  particularly  if  the  finalized  training 
intervention  is  not  yet  available.  Additionally,  administering  the  entire  AXL  intervention  (both 
films  and  a  minimum  of  four  modules)  would  require  a  large  time  commitment  from  Soldiers, 
taking  valuable  time  away  from  other  important  leadership  activities. 

The  researchers  have,  therefore,  adopted  an  iterative  evaluation  process  to  test  various 
components  of  the  AXL  system  at  a  time.  When  the  first  beta-version  of  AXL  was  produced,  the 
researchers  examined  differences  due  to  media  effects  (Zbylut  &  Ward,  2004a;  Zbylut  et  ah, 
2005).  Testing  for  media  effects  allowed  the  researchers  to  conduct  studies  that  required  a 
limited  number  of  research  participants  for  a  few  hours.  This  permitted  the  researchers  to 
observe  how  leaders  discussed  an  AXL  case  study  when  an  instructor  was  not  present,  as  well  as 
gather  reactions  to  different  modes  of  presenting  the  material.  Observations  were  then  used  to 
refine  the  AXL  interface  and  make  modifications  to  the  pedagogical  approach.  In  sum,  the  first 
wave  of  data  collection  primarily  targeted  usability  issues,  media  effects,  and  reactions  to  the 
AXL  approach. 

The  results  reported  in  this  paper  represent  the  second  iteration  of  AXL  evaluation.  This 
second  wave  of  data  collection  focused  on  gathering  reactions  to  the  revised  AXL  approach, 
assessing  learning  within  the  context  of  a  single  AXL  module,  and  exploring  the  relationship 
between  emotions  and  learning.  Similar  to  our  previous  experience,  only  a  limited  number  of 
leaders  were  available  for  participation  and  only  two-and-a-half  hours  were  allotted  for  data 
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collection.  Given  the  time  constraints  and  the  desire  to  have  a  sample  large  enough  to  conduct 
statistical  tests,  the  researchers  chose  to  focus  evaluation  efforts  on  a  single  AXL  module  and 
evaluate  learning  within  the  constraints  of  a  short-term  context. 

Evaluating  learning  within  a  short-term  context  presented  a  research  challenge  for  this 
particular  project.  Although  a  portion  of  AXL  interactive  modules  is  devoted  solely  to  the 
analysis  of  the  film,  the  AXL  system  also  requires  that  leaders  reflect  on  and  discuss  their 
personal  experiences  and  make  comparisons  between  their  experiences  and  the  film.  Moreover, 
it  is  likely  that  the  sharing  and  discussion  of  these  leadership  experiences  is  one  of  the  more 
valuable  aspects  of  the  AXL  system  (Palus,  Horth,  Selvin,  &  Pulley;  2003;  Zbylut  &  Ward, 
2004a).  Unfortunately,  since  each  leader’s  experience  is  unique  and  group  dynamics  can  take  a 
discussion  in  a  multitude  of  different  directions,  it  is  difficult  to  predict  precisely  what  “pearls  of 
wisdom”  leaders  will  extract  from  AXL  instruction. 

We  dealt  with  the  challenge  of  assessing  learning  in  a  short-term  context  in  a  few 
different  ways.  The  simplest  approach  was  to  gather  self-reports  of  learning.  We  asked  leaders 
to  indicate  whether  the  instructional  approach  was  thought-provoking  and  also  if  they  would  be 
able  to  transfer  something  that  they  learned  to  their  activities  as  a  leader.  The  other  two 
approaches  used  to  assess  learning  were  a  rank-ordering  task  and  a  set  of  forced-choice  items 
targeting  judgment.  Both  of  these  approaches  required  us  to  conduct  a  preliminary  data 
collection  effort  in  order  to  ascertain  what  issues  the  target  training  audience  found  salient  and 
what  courses  of  action  the  target  training  audience  thought  were  appropriate  in  the  Tripwire 
scenario.  While  all  of  these  short-term  approaches  taken  together  provide  an  indication  that  the 
AXL  system  holds  promise,  the  primary  question  that  remains  unanswered  is  how  the  approach 
impacts  general  knowledge  and  leader  behavior  over  time.  Access  to  sufficient  numbers  of 
officers  in  a  longitudinal  context  is  required  to  answer  that  question. 

Concluding  Remarks 

Practitioners  interested  in  the  delivery  of  online  leader  development  tools  can  easily 
leverage  many  of  the  techniques  applied  in  the  AXL  system.  Specifically,  using  multimedia  case 
studies,  particularly  in  film  format,  appears  to  capture  leader  attention  and  evoke  emotion. 
Moreover,  a  computer  can  be  used  to  facilitate  the  group  discussion  process  through  the  use  of 
open-ended  questions  that  target  analysis  of  the  case  study  and  reflection  on  personal 
experiences.  Close-ended  questions  can  be  used  to  branch  discussion  along  different  lines  of 
thought  and  provide  feedback  to  individuals  to  exert  quality  control  over  the  discussion.  This 
paper  provides  evidence  that  online  instructional  tools  can  be  both  interesting  and  educational. 
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